(12) 



UK Patent Application „s,GB ,,,,2374225 ,,3, A 



(43) Date of A Publication 09.10^002 



(21) Application No 0107805.4 

(22) Date of Filing 28.03.2001 



(71) Applicant(s) 

Hewlett-Packard Company 

(Incorporated in USA - Delaware) 

3000 Hanover Street, Palo Alto, California 94304, 

United States of America 

(72) Inventor(s) 

Stephen Philip Cheatle 
David Arthur Grosvenor 
David NeU Slatter 

(74) Agent and/or Address for Service 
Richard Anthony Lawrence 
Hewlett-Packard Limited IP Section, RIton Road, 
Stoke GIfford, BRISTOL BS34 8QZ, United Kingdom 



(51) INTCL^ 

H04N 1/32 // H04N 5/445 

(52) UK CL (Edition T ) 

H4F FBB FGG F12S F22 

(56) Documents Cited 

EP 0820179 A2 US 6133947 A 

US 5267042 A 

(58) Field of Search 

UK CL (Edition S ) H4F FAAX FBB FGG FGH 
Online: WPL EPODOC, JAPIO 



(54) Abstract Title 

Camera for recording linked information associated witli a recorded image 

(57) Apparatus for providing composite electronic image of a scene including at least one link to other 
information comprises a camera 1 for providing an image signal, link generating means responsive to the 
scene or the immediate environment for generating a link signal, and means for combining the image and link 
signals to provide said electronic image. As shown, the viewed scene includes a zoo information display 7 with 
a barcode which is read by the camera to provide the final |Mrt of the link Information for concatenation onto a 
first part which loaded into the camera at entry to the 200 (or preloaded if the camera belongs to the zoo). 
Clicking on the display 7 on the viewed image provides a link to a zoo web page associated with the 
enclosure 4. 
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Recording Images Together with Link Information 

The present invention relates to the production of a composite signal comprising an 
image signal together with a link information signal providing a director to additional 
information. In use the link information signal may be accessed for example by 
clicking a cursor on a predetermined area of, or the whole of, the image. 

This type of composite signal occurs, for example in the Windows environment on a 
PC, where there is an underlying functional map corresponding to the image area 
(sometimes referred to as metadata), and where selected areas will cause a cursor to 
alter in a predetermined manner, commonly a change in shape, and where clicking on 
the changed cursor will cause the PC to effect a function as determined by the 
functional map. This also occurs in web pages, where the function is commonly to 
redirect the computer to a different web file, as in an HTML. Effectively both image 
(pixel) data and functional data (link information) are associated with the same 
(normally local) display area. The pixel information is displayed by the PC, when the 
mouse position falls within the local area the PC responds by altering a cursor 
characteristic, and if the mouse is then clicked the PC proceeds to operate upon the 
associated functional data. 

Still and moving images are increasingly produced from representative digital signals, 
for example as part or the whole of a web page, and web pages commonly include 
links to other web pages or other files. Where a fiill address is given by the link, this 
may be the URL (Uniform Resource Locator) to a web page. In other instances, the 
web page is one of a set having a common home address (an aforesaid supplementary 
link signal), and the link proves additional address information which when 
concatenated with the home address provides the full URL. 

Web design tools are known which enable a web author to put images on web pages 
and manually to create one or more "hot-spots" on the image which are hyperlinked to 
other web pages. Not only is this process tedious, but the author needs to know the 
destination web page when the "hot-spots" are created. 
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It is also known to use a global positioning system (GPS) to associate location data 
with a photograph, typical examples being described in US Patent No. 6,091,956 
(Hollenberg) and US Patent No. 5,267,042 (Tsuchiya). In certain instances, the 
location data is subsequently used to provide additional information. Thus in US 
Patent No. 6,133,947 (Mikuni) the location data is used to relate the camera image to 
a map. 

In US Patent No. 5,926,1 16 (Kitano) the location data is used for image retrieval from 
a local Web database comprising images which are associated with respective 
locations and URLs; matching of location data and image data provides the 
corresponding URL so that further information related to the image in question can be 
obtained at the point of use, and no further use or storage of the image data appears to 
be envisages - it is merely used for the purposes of web access. 

The present invention facilitates the automatic generation of one or more links from a 
camera image, or fix)m at least one area of the image, to further information. In its 
broadest aspects, by contrast with Kitano it does not require immediate knowledge of 
absolute location as provided by a GPS system; nor does it require access to a local 
database such as a Web database, nor access to any pre-storcd images for 
confirmation of the required web address. Furthermore the image matching step of 
Kitano is usually a very difficult process, and is not necessary to the implementation 
of the present invention in its broadest aspect, where all that is required for generating 
the link information may be, for example, the reading of a barcode or an ASCII 
sequence. 

The present invention provides apparatus for providing a composite signal comprising 
an image signal together with a link information signal providing a director to 
additional information, the apparatus comprising a camera for providing an image 
signal, link signal means responsive to the scene or the immediate environment for 
generating a said link information signal, and signal combining means for combining 
the image and link information signals to provide said composite signal. 

Preferably the link signal means or at least a part thereof is part of the camera. Where 
it is necessary to access a remote or local database, using the locally provided link 



infoimation signal, to obtain a further address (a further link information signal), this 
may be effected at the time of taking the picture for the purpose of combining the 
further address with the image signal, and in this case the camera will need to be 
adapted to interrogate the database in real time. However, preferably it is the locally 
5 provided link information signal which is combined with the image signal and access 
to any necessary database is effected later on the basis of the stored composite signaL 

Preferably the signal combining means is part of the camera. 

The image signal may be for example a recording of a still image, e.g. a photograph as 
produced by an electronic camera, or live or recorded video i.e, of a moving image as 
10 viewed by a video camera. The photographs or video clips may be provided as an 
album. 

In the case of video clips, these may be actuated from the album, which may show 
just the starting frame initially. Also the link information may be arranged to be 
executable either on a still image from the video, or the moving image. Where the 
15 location of the link information moves with the viewed image, i.e. a local "hot spot" 
as opposed to a fixed icon, this may be effected using standard tracking techniques. 

The link information signal may be any signal providing a full or partial address to a 
source of data or information, for example a required file location, and the file 
location may be local or remote. However, it could also be a differently coded signal 
20 which may be converted to a partial or complete address for example by the use of an 
appropriate local or remote database or look-up table. 

Where the address provided by the link information signal is only a partial address, 
the fiill address to the required location may be implicit, or it may be provided by 
combining it in a known or predetermined manner with a supplementary link signal. 
25 The latter may be predetermined, or a variable determined by the conditions of use. 

Thus in the case which depends firom a home page as described above, the 
supplementary link signal may be the home address which is a variable dependent on 
the set of web pages under consideration, but otherwise predetermined by that set. 
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In another example a site owner (for example of a theme park where there are a 
number of rides, or another public venue where there are a number of exhibits) may 
also own a website. The link information may be supplied by means responsive to the 
scene or the immediate environment as appropriate, as outlined in more detail below, 
but may only provide a partial address or address information, in this case a partial 
URL needing to be supplemented by the home website address. There may then be a 
facility for programming the visitor's camera with the home address (for example 
when paying a fee for admission to the site), which is thus variable supplementary 
link information dependent on the site being visited, Altematively, where a camera is 
provided by site owner this may be pre-programmed with the home address of the 
website, which is therefore wholly predetermined supplementary link information. 
Indeed, if used on-site with facilities provided by the site owner, for example locally 
stored computer files, the remainder of the address may be implicit and may never 
actually need to be used. 

Location data, for example as provided by the aforementioned US Patents may also be 
combined with the electronic image for providing a further link information signal. 
However, where the means for providing such location data involves a GPS, this is 
not regarded as being means responsive to the scene or the inmiediate environment, 
since GPS requires a transponder for active interrogation and response to establish 
location, and the transponder is responsive neither to the scene or its inunediate 
environment, only to the location thereof as determined by the GPS. 

In general the means responsive to the scene or the immediate environment will 
include means responsive to a feature within the scene or environment. A number of 
different features can be utilised, according to the intended method and location of 
use of the apparatus. Examples include: 

1. Object recognition means, which recognises when a target having a 
predetemiined characteristic such as form, shape or motion (for example a person's 
face, or a vehicle such as a car, or a recognisable logo) enters or is in the field of view 
of the camera. 

2. Means for reading encoded patterns in the scene where the target is 
expected to bear such a pattern. Exemplary patterns include a bar code or ASCII 



URL provided on a badge worn by a person attending an event or visiting a public 
place such as an amusement park, or on a sign or information board at an exhibit, or 
on a web camera (e.g. also viewing the same or a related image); or a car number 
plate. 

5 3. Means for responding to a remote transmitting beacon, or for interrogating a 

transponder, carried by the target, e.g. as in 2 above. The necessary one-way or two- 
way communication may be via any known means, such as radio, infra-red, or ultra- 
sound, and the same or different ones of these methods may be used for two-way 
communication as desired. 

10 The link information signal may be directly generated in response to detection of 
feature(s) within the scene or environment. Thus in option 2 in the preceding 
paragraph, the location and information in the bar code or ASCII URL may be 
sufficient in itself to be translated into the hyperlink with an appropriate location in 
the resulting electronic image. 

15 In other cases, it may be necessary to look up stored information in a database to 
associate the detected feature(s) with a corresponding hyperlink or URL, which is 
then added to the stored image at an appropriate location, and the means responsive to 
the scene or the immediate environment will include means for interrogating the 
database for retrieving the associated hyperlink or URL information. In such a case, 

20 the database may be local or remote, for example provided by a publicly available 
service. 

As an example, a person attending an event may be recognised from a local database 
prepared by assembling images of each person attending and suitable address 
information including link information, e.g. the person's web address. When a person 
25 is recognised as being viewed by a camera, the database is inten-ogated to retrieve the 
associated link or URL information, which is then associated with the location of the 
person in the electronic image, or with the entire image. 

When at least part of the link information is derived from a person (e.g. recognition or 
reading a tag) the link may be to provide personal details, such as a mailing address or 
30 e-mail address. There may be a facility for pn-site recordal of the composite signal to 
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facilitate actions at a later date, possibly computed to occur a predetermined interval 
after the date at which the original image was taken, such as issuing reminders about 
the site to the person involved, and/or details of forthcoming events. 

When at least part of the link information is derived from a predetermined location, 
5 the link may be to provide further details about the location. For example where an 
information board for an exhibit also provides the link information e.g. as a bar code, 
the board information and/or other educational details about the exhibit or site may 
then obtained over the web, including historical details or images. Additionally or 
alternatively there may be provided access to advertising; to details about admission 
10 charges and times; to details concerning future exhibits (e.g. type and timing); and/or 
to on-line shops associated with the location. 

The predetennined location may also be on, or associated with a web camera, for 
example one also viewing the image at the time of taking the picture. Again this may 
be indicated by an icon in the reproduced picture so that the camera may be accessed 
15 at a later date to revisit the location over the web, so that, for example, the growth of - 
an animal (web camera at a zoo) or the development of a building site (web camera at 
a viewing platform of a development) may be followed. 

Where a logo in the enviromnent is recognised, there may be automatic generation of 
a link to the site of the logo owner, for example for the purposes of advertising by the 
20 logo owner and revenue from the owner of the site where the logo is located, or for 
links to related attractions. 

Further link information may be added to the electronic image to associate that image 
with other images taken closely together in time, for example other images of the 
same target taken by the same camera, or, by consulting a local or remote database 

25 receiving information from a plurality of cameras distributed over a site, other images 
of the same target taken by different cameras. Where images contain the same link 
information, they may be grouped to each other by a further link. Not only can this 
provide a customer with a set of related images, but it can facilitate tracking of the 
customer and the provision of other information to the manager of the site. In such a 

30 case, there may be a facility for inserting in the viewed image an indication that the 



link exists, for example as a clickable icon at the foot of the picture (being associated 
with the link information in the image functional map). 

The or one link may be associated with substantially the entire image, but in general a 
link will be associated with a particular location or small area within the image. The 
5 latter may be arranged to over-ride any link associated with the entire image. 

In general it may be desirable to include an icon in the viewed image to indicate 
where a link exists. The placement of such an icon may be predetermined, or, 
preferable, placed within a relatively uninteresting area of the image as determined, 
for example, by analysis of the image. However, in certain instances it will be 
10 possible to associate the link with the area of or including an identifiable object in the 
image, for example a car or an information board, or a logo, such areas being 
determined for example by analysis of the image signal. 

As thus far discussed, the link information is derived fix)m the scene when taking' the 
picture. However it is also possible to capture the link information separately for 

15 combination with a picture taken earlier or later. Thus, taking the example of a visitor 
at an exhibit with an information board, a control may be provided on the camera 
whereby a link may be read by the camera immediately before or after (the control 
may provide selection as to which, or it may be predetermined) a picture of the 
exhibit is taken, the link information sign2il and image signal being combined to 

20 provide the composite signal. 

As already noted, the link information need not be generated by optical recognition of 
a feature in the scene or environment. It is also possible to arrange for active local 
transmission of link information to a dedicated camera, for example by radio, infra- 
red or inductive linkages, or by a transponder responsive to a signal from the camera. 
25 In such a case, the camera may provide an indication to the user when link 
information is present, and there may further be the choice of whether of not to 
include the information in any picture. Thus, again taking the example of a visitor at 
an exhibit, there may be an infra-red transmitter of link information in the vicinity of 
the exhibit to which the camera responds by illuminating an LED externally and/or in 
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the camera viewfinder. When the visitor takes a picture of the exhibit the link 
information signal in combined with the image signal. 

Again, direct signal means may provide at least part of the link information. For 
example a visitor may place the camera in a dedicated receptacle at the location, 
where electrical contact is made to the camera and information passed thereto, e.g. by 
the transmission of a signal containing the link information, or by the use of coded 
contacts with an arrangement peculiar to the location. 

Further features and advantages of the invention will become clear upon consideration 
of the appended claims, to which the reader is referred, and upon a reading of the 
following more detailed description of an exemplary embodiment of the invention, 
made with reference to the accompanying drawing, in which Figures 1 and 2 
schematically illustrates an embodiment of the invention for use in connection with a 
person viewing a number of exhibits. 

Figure 1 shows a person (visitor) 1 who has paid to enter a zoological park (zoo), and 
who has been provided with an electronic camera 2, which contains predetermined 
link information relating to the base URL for the zoo. The zoo has a website with a 
series of pages relating to different exhibits/enclosures, the link information for each 
page commencing with the base URL. 

Visitor 1 is using camera 2 to take a photograph of a scene which includes another 
visitor 3 standing looking into an enclosure 4 where an attendant 5 is feeding an 
animal 6. In front of the enclosure and within the view of the camera 2 is located an 
infomiation display 7 about the animal in the enclosure. 

The display 7 also includes a bar code which is recognised and read by the camera to 
provide link information. This is concatenated onto the end of base information to 
provide a fiiU hypertext Imk or URL associated with a hot-spot in the region of the 
barcode in the resulting electronic image. Alternatively an icon associated with the 
URL may be provided at a predetermined location such as a comer of the image. 

When the image is later inspected on a viewer 8, Figure 2, clicking on the area of the 
barcode 9 (or icon) causes the hyperlink or URL 10 to be traversed to obtain the web 
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page 1 1 relating to the enclosure, for example educational detail regarding the animal 
6. 

In this example the camera 2 is retumed to the zoo at the exit, with images on disc 
being given to the visitor 1. Clearly, it would also be possible for the visitor 1 to own 
5 a camera 2 which is loaded with the base URL on admission to the zoo, with image 
retrieval after the visitor has left the zoo. 

Where the camera remains with the zoo, it may be appropriate for the base 
information to be permanently stored in the camera, so that a stolen camera is 
effectively of reduced use in other locations. However, storage may be temporary and 
10 re-writable, and this would be necessary for a camera belonging to a visitor. 

It should be appreciated that the image signal may be transmitted to a remote location 
for combination with the link information signal and recordal rather than at the 
camera, for example at a central computer in an installation where there are a plurality 
of cameras all linked to the computer. 

15 Also the means providing the link information signal is not necessarily part of the 
camera. Thus for example where the image signal is sent to a central computer the 
link information signal there may be means at a particular location for identifying the 
camera and transmitting the identification information to the central computer 
together with associated link information, e.g. relating to the particular location, etc. 
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CLAIMS 



1. Camera apparatus for providing a composite signal comfffising an image 
signal together with a link information signal providing a director to additional 
information, the apparatus comprising a camera for providing an image signal, link 
signal means responsive to the scene or the immediate environment for generating a 
said link information signal, and signal combining means for combining the image 
and link information signals to provide said composite signal. 

2. Camera apparatus according to claim 1 wherein the link signal means is 
responsive to coded information, or a beacon, or a transponder, or a direct signal input 
means within the scene or the immediate environment for directly or indirectly 
providing at least part of the link information signal. 

3. Camera apparatus according to claim 1 or claim 2 wherein the link signal 
means is arranged for recognising or reading an optical feature or features in the scene 
or the inunediate environment and for associating therewith at least part of the link 
information signal. 

4. Camera apparatus according to any preceding claim whaein the link signal 
means includes means for analysing said image signal for recognismg therefrom an 
optical feature or features in the scene or the immediate environment, and for 
associating therewith at least part of the link infonnation signal. 

5. Camera apparatus according to any preceding claim wherein part of the link 
information signal is predetermined information, and the link signal means includes 
means for temporarily or permanently storing such predetermined information. 

6. Camera apparatus according to any preceding claim wherein the signal 
combining means includes means for associating at least part of said link information 
signal with a local area, spot or icon in the image. 

7. Camera apparatus according to claim 6 including means for (tetermining said 
local area by analysis of the image signal. 
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8. Camera apparatus according to claim 7 claim wherein the said associating 
means is arranged to locate a said icon at a predetermined position in the image. 

9. Camera apparatus according to claim 7 wherein the said associating means is 
arranged to locate a said icon at a position in the image determined by analysis of the 
image signal. 

1 0. Camera apparatus according to any preceding claim wherein at least one part 
of the link information signal is associated with substantially all of the image, barring 
any local areas associated with other parts of the link information signal. 

11. Camera apparatus according to any one of claims 6 to 10 wherein there are at 
least two said local areas, hot spots, or icons. 

12. Camera apparatus according to any preceding claim v^erein further link 
information is generated from information regarding the location of the scene. 

13. Camera apparatus according to any preceding claim wherein further link 
information is generated for linking images of the same target taken by the same 
camera, or by different cameras operating on the same site. 

14. Camera apparatus according to any preceding claim wherein the link signal 
means is included in the camera. 

15. Camera apparatus according to any preceding claim wherein the signal 
combining means is included in the camera. 

16. Camera 2q>paratus according to any preceding claim wherein the camera has a 
facility for recording a said link signal at a time when an image signal is not being 
recorded and for combining it with an image signal taken at another time, 

17. A method of generating a composite signal including the steps of generating 
an image signal corresponding to a scene, generating a link information signal 
providing a director to additional information in response to the scene or the 
immediate environment, and combining the image and a link information signal to 
provide said composite signal. 
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18. A method according to claim 17 including the step of analysing the image 
signal to determine a said link information signal. 

19. A method according to claim 17 or claim 18 and including the step of 
receiving a said link information signal from an emitter, transponder, or direct signal 
input associated with the scene or the immediate environment, 

20. A method according to any one of claims 17 to 19 and including the step of 
using the link information signal to interrogate a remote or local database to provide a 
further link information signal for combination in said composite signal. 

21. A method according to any one of claims 17 to 20 and including the step of 
combining a said link information signal with a predetermined signal for use as a said 
link information signal to be combined with said image signal. 

22. A method according to claim 21 wherein said predetermined signal is recorded 
once before a plurality of said composite signals are recorded. 

23. A method according to claim 21 or claim 22 wherein said predetermined 
signal is provided as a base hyperlink signal, 

24. A method according to claim 23 wherein said link information signal 
generated in response to the scene or the immediate environment is complementary to 
said base signal and is combined therewith to produce a full URL. 

25. A method according to any one of claims 17 to 20 wherein said link 
information signal generated in response to the scene or the inmiediate environment is 
a full URL. 

26. A method according to any one of claims 17 to 26 wherein a composite signal 
is provided with a further link to another composite signal taken by the same user, or 
an identical link information signal. 

27. A method according to any one of claims 17 to 20 wherein a said link 
information signal is provided as an e-mail address. 
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28. A method according to claim 27 and including the step of transmitting 
information to the e-mail address. 

29. An installation for use with camera apparatus according to any one of claims 1 
to 16 and including a plurality of said cameras, and including means for recording the 
composite signals from each camera at a common location. 

30. An installation according to claim 29 wherein the recording means includes 
means for grouping the composite signals according to at least one said link 
information signal. 
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